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ABSTRACT.  Due  to  the  large  number  of  entities  and  processes  that  must  be  represented,  combat 
models  at  the  theater  level  in  the  Army  today  are  expected  value  models.  An  expected  value  model  is 
deterministic  -  it  uses  the  expected  value  of  random  variables  as  inputs  and  generally  uses  some  sort 
of  expected  value  within  the  internal  processes.  The  use  of  expected  value  models  creates  problems  in 
the  proper  interpretation  of  their  output  and  ways  for  representing  the  uncertainty  associated  with 
the  model  input  and  processes. 

This  paper  suggests  a  method  for  handling  uncertainty  in  the  input  data  sets  (which  usually 
contain  elements  that  are  specific  realizations  of  random  processes)  in  situations  where  the  outcomes 
of  interest  can  be  expressed  in  binary  variables  (e.g.,  “success”  or  “failure”).  A  theater  nuclear 
exchange  is  used  as  an  example,  having  many  different  possible  outcomes  determined  by  random 
processes.  A  method  is  provided  for  describing  the  space  of  all  possible  outcomes  of  the  exchange  and 
partitioning  the  space  into  sets  of  outcomes  which,  if  used  as  input  into  a  theater-level  conventional 
simulation,  are  expected  to  lead  to  significantly  different  results.  A  method  for  sampling  the  most 
probable  outcome  from  each  set  is  also  explained. 

This  approach  permits  the  construction  of  an  experimental  plan  that  requires  a  small  number  of 
model  runs,  each  run  expected  to  provide  a  significantly  different  result.  From  these  runs  an 
estimate  of  the  variability  in  the  theater  combat  resulting  from  uncertainty  in  the  input  data  (in 
this  case,  the  impact  of  a  nuclear  exchange)  can  be  made. 


THE  RESEARCH  SPONSOR  was  the  Director,  US  Army  Concepts  Analysis  Agency  (CAA). 

THE  OBJECTIVE  OF  THE  RESEARCH  was  to  develop  a  method  for  summarizing  a  stochastic 
process  as  input  into  a  deterministic  expected  value  model. 

THE  MAIN  ASSUMPTIONS  used  in  this  research  were: 


(1)  The  stochastic  process  (outcome  of  a  nuclear  exchange)  can  be  summarized  os  sets  of  binary 
variables,  each  variable  indicating  the  defeat  or  failure  to  defeat  a  unit. 


(2)  It  is  possible  to  identify  significantly  different  outcomes  of  a  nuclear  exchange  in  terms  of 
seta  of  defeated  and  nondefeated  units. 

(3)  The  probabilities  of  defeatiug  units  can  be  evaluated  independently. 

THE  BASIC  APPROACH  used  in  this  research  was  to  use  the  probability  that  a  targetable  subunit 
(such  as  a  company  or  battery)  can  be  defeated  (which  can  I*  obtained  from  the  Theater  Analytic 
Nuclear  (TACNUC)  Model  under  development  at  CAA)  to  determine  the  probability  that  units 
(such  as  divisions)  represented  in  the  theater-level  expected  value  model  can  be  defeated.  The  space 
of  all  possible  outcomes  of  the  exchange  is  described  in  terms  of  binary  variables  representing  the 
defeat  or  failure  to  defeat  a  unit.  The  outcome  space  can  be  partitioned  into  sets  of  outcomes  which, 
if  used  as  input  into  a  theater-level  conventional  simulation,  will  lead  to  significantly  different 
results.  From  these  partitions  an  experimental  plan  is  constructed  that  identifies  a  small  number  of 
outcome  realizations  of  the  exchange  to  use  as  input  to  the  expected  value  model,  running  the  model 
once  for  each  input. 
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THE  PRINCIPAL  FINDING  of  the  research  is  that  is  is  possible  to  summarize  a  stochastic  process 
such  as  the  outcome  of  a  theater  nuclear  exchange  as  input  to  a  deterministic  conventional  model  of 
battle  at  the  theater  level. 

THE  RESEARCH  WAS  PERFORMED  BY  MAJ  Mark  A.  Youngren,  Requirements  Directorate, 
CAA. 

COMMENTS  AND  QUESTIONS  may  be  sent  to  the  Director,  US  Army  Concepts  Analysis  Agency, 
ATTN:  CSCA-RQN,  8120  Woodmont  Avenue,  Bethcsda,  MD  20814-2797. 
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HANDLING  UNCERTAINTY  IN  INPUT  TO  EXPECTED  VALUE  MODELS 


Introduction 

Modeling  large  systems  and  processes  such  as  combat  at  the  theater  level  is  difficult.  The 
number  of  possible  units  and  interactions  has  driven  most  modelers  to  use  an  expected  value 
approach.  An  expected  value  model  uses  the  expected  value  of  random  variables  as  inputs  and 
generally  uses  some  sort  of  expected  value  within  the  internal  processes.  The  models  are 
deterministic ;  that  is,  they  will  yield  only  one  set  of  outputs  for  any  given  set  of  inputs.  The  use  of 
expected  value  models  creates  problems  in  the  proper  interpretation  of  their  output  and  ways  for 
representing  the  uncertainty  associated  with  the  model  input  and  processes.  In  a  recent  discussion 
paper,  Stockton  [1989]  provided  the  following  example: 

“A  Red  unit  will  go  northwest  or  northeast  based  on  whether  his  strength  at  a  given  point  is 
above  or  below  some  threshold  value.  Let’s  say  that  the  real-world  probability  of  being  above  the 
threshold  is  0.6  and,  if  above,  he  will  go  northwest  to  face  a  very  strong  Blue  force  armed  with 
Supertank.  If  he  goes  northeast  (probability  0.4),  he  faces  a  relatively  weaker  force,  armed  with  bows 
and  arrows.  With  several  replications  of  a  stochastic  model,  expected  losses  will  consider  both 
possibilities  and  will  develop  expenditures  of  tank  ammo  and  arrows;  with  an  expected  value  model, 
he  will  always  go  toward  the  stronger  force,  and  no  expenditures  of  arrows  will  be  observed.” 

Stockton  correctly  points  out  that  the  results  of  an  expected  value  model,  even  when  provided 
expected  value  inputs,  are  not  the  expected  value  of  the  output.  He  suggests  that  the  output  of  such 
a  model  may  be  a  “most  likely  value,”  using  his  example.  However,  we  can  offer  another  example 
which  illustrates  that  expected  value  models  also  fail  to  provide  a  “most  likely”  result. 

Suppose  in  the  example  provided  above  that  the  Red  force  has  a  visual  sensor  that  can  see  all  of 
the  Blue  forces  traveling  together  (with  probability  1)  if  the  skies  are  clear,  and  cannot  see  any  of 
the  Blue  force  if  the  skies  are  cloudy.  To  simplify,  suppose  that  the  skies  are  either  clear  or  cloudy, 
and  the  probability  that  the  skies  are  clear  is  0.6.  How  many  Blue  units  are  detected  by  the  Red 
force?  The  expected  value  is  0,6  •  (100  percent  of  the  Blue  units)  +  0.4  •  (0  percent  of  the  Blue 
units)  =  60  percent  of  the  Blue  units.  Expected  value  models  will  normally  apply  expected  values, 
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either  as  inputs  to  the  model  (60  percent  would  be  an  expected  value  for  the  probability  of  target 
acquisition)  or  internal  to  the  processes,  Note,  however,  that  acquiring  60  percent  of  the  Blue  force  is 
the  least  likely  outcome,  as  it  occurs  with  probability  0!  Even  if  we  chose  the  most  likely  result  of 
100  percent  detection  (which  is  not  the  way  that  expected  value  models  generally  handle  continuous 
variables  as  opposed  to  choices),  we  run  into  problems. 

Now  let  us  combine  the  two  examples.  It  is  reasonable  to  suppose  that  if  the  Red  force  can  see 
the  Blue  force,  or  even  a  large  percentage  of  the  force,  it  will  notice  that  one  force  is  armed  with 
Supertank  and  the  other  with  bows  and  arrows.  Thus,  given  detection,  it  will  engage  the  weaker 
(bows  and  arrows)  force.  If  we  have  the  model  take  the  most  likely  values  in  the  two  examples,  it 
will  (1)  detect  100  percent  of  the  Blue  force  and  (2)  go  northwest  to  engage  the  Blue  force.  Each 
result  is  by  itself  most  likely,  yet  the  result  is  the  most  unlikely.  Even  if  one  modeled  the  Red  force 
detection  at  60  percent,  the  combination  of  a  60  percent  detection  (still  sufficient  to  distinguish 
between  Supertank  and  bows  and  arrows)  and  moving  northwest  is  unlikely. 

Admittedly,  these  examples  are  simplistic.  Yet  it  is  true  that  expected  value  models  not  only  fail 
to  yield  the  expected  value  of  the  output,  they  also  fail  to  yield  the  most  likely  output.  What,  then, 
is  the  probability  associated  with  the  output  of  an  expected  value  model?  The  answer  to  that 
question,  unfortunately,  is  “nobody  knows.”  This  is  why  expected  value  models  can  yield 
counterintuitive,  contradictory,  and/or  nonsensical  results  when  initially  tested.  The  usual  approach 
when  this  occurs  is  to  adjust  input  data,  processes,  thresholds,  etc.  until  the  model  yields 
“reasonable”  results.  Hopefully  this  yields  a  model  that  will  provide  suitably  realistic  results  with  a 
different  input  data  set,  but  there  are  no  guarantees.  We  unquestionably  have  no  way  of  determining 
the  likelihood  of  any  given  output  from  a  complex  expected  value  model. 

Sources  of  Uncertainty 

There  are  two  areas  of  uncertainty  properly  associated  with  an  expected  value  model  that  must 
be  handled:  uncertainty  in  the  model  input,  and  uncertainty  in  the  model  processes. 

Unfortunately,  a  “blessed”  input  data  set  is  often  regarded  as  certain  -  if  we  have  approval  for  a 
set  of  numbers  to  be  used  in  the  study,  then  those  numbers  are  Ure  set  to  use  to  support  our 
analysis.  Excursions  from  the  base  data  set  for  purposes  of  analysis  will  vary  only  a  small  number  of 
data  items  by  design;  the  others  remain  fixed.  Some  input  data  values  are  truly  fixed;  the  air 
distance  from  Bremen  to  Munich  is  an  example.  Other  values  may  be  fixed  by  scenario;  for  example, 
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the  daylight  hours  vary  by  latitude  and  time  of  year;  a  scenario  will  fix  a  time  and  place  that  will  in 
turn  determine  the  u|  propriate  value  for  daylight.  Unfortunately,  these  scenario-driven  items  are 
often  fixed  arbitrarily,  even  when  they  may  have  an  impact  upon  the  analysis.  For  example,  if  a 
force  is  particularly  vulnerable  to  detection  by  a  sensor  that  requires  daylight,  you  can  get  different 
results  in  a  summer  versus  winter  scenario  (which  will  in  turn  be  different  than  that  obtained  using 
an  arbitrary  number  like  8  hours  or  12  hours).  This  difference  may  even  be  apparent  in  studies  that 
seemingly  are  not  associated  with  detection  --  ammo  rates  could  be  significantly  different,  for 
example.  This  is  a  simple,  obvious  example;  many  others,  not  so  easily  identified,  exist.  We  must 
regard  the  input  data  set  as  a  single  realization  of  many  stochastic  variables.  It  is  not  always  clear 
which  rea'ization  to  select  for  use  --  averages  do  not  always  exist  and  may  not  be  appropriate. 
Furthermore,  correlations  exist  between  sets  of  these  data  inputs;  for  example,  selecting  the  most 
likely  or  expected  values  of  cloud  cover  and  rain  independently  may  yield  the  combination  of  sunny 
with  1  inch  of  rain!  Note  that  this  problem  exists  with  stochastic  (Monte  Carlo)  models  —  they  also 
require  a  fixed  data  set  that  is  net  varied  from  run  to  run. 

Uncertainty  also  exists  in  th;  model  processes.  Stochastic  models  generally  handle  this 
uncertainty  through  random  number  draws,  although  they  are  also  subject  to  problems  associated 
with  correlations  (separate  random  number  draws  generally  require  independence)  and  fixed  values 
such  as  thresholds.  The  examples  provided  above  illustrate  some  of  the  problems  associated  with 
handling  process  and  input  uncertainty  within  an  expected  value  model. 

Addressing  Uncertainty  in  Expected  Value  Models 

At  this  point,  it  would  be  nice  to  be  able  to  make  a  statement  like  “the  solution  to  this  problem 
is  easy;  one  simply  needs  to...  .”  Unfortunately,  there  are  no  simple,  universal  solutions  to  the 
problems  associated  with  addressing  uncertainty  in  expected  value  models.  It  is  clear,  however,  that 
any  methods  that  might  alleviate  the  problem  must  deal  with  the  uncertainty  associated  with  the 
data  input  as  well  as  the  uncertainty  associated  with  the  model  processes.  Furthermore,  the 
uncertainty  in  the  input  data  justifies  the  following  assertion:  executing  an  expected  value  mode!  only 
once  for  a  given  data  set  does  not  provide  a  meaningful  result.  If  an  expected  value  model  is  to  be 
used  to  support  analysis,  the  user  must  be  prepared  to  execute  multiple  runs,  varying  in  some 
meaningful  fashion  the  input  data  and/or  the  model  processes,  in  order  to  establish  some  measure  of 
the  uncertainty  associated  with  the  output  of  such  a  model. 
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Ideally,  such  an  approach  will  minimize  the  number  of  runs  required  (because  running  a  large 
expected  value  model  can  be  very  costly),  yet  provide  a  significantly  different  result  from  each  run, 
thus  increasing  the  variance  across  all  outputs,  We  want  to  be  able  to  describe  the  probability  that 
the  conditions  represented  in  the  input  for  each  run  (or  conditions  similar  to  those  represented)  will 
occur. 

We  have  developed  an  approach  to  handling  input  uncertainty  in  theater-level  expected  value 
models  in  situations  when  the  outcomes  of  interest  can  be  expressed  in  terms  of  binary  variables; 
i.e.,  one  can  describe  all  events  as  “yes”  or  “no,”  “on”  or  “off,”  etc,  The  particular  application  that 
will  be  developed  deals  with  a  theater-level  tactical  nuclear  exchange. 

Several  models  of  conventional  warfare  exist  at  the  theater  level.  The  model  used  at  CAA  is 
called  the  Force  Evaluation  Model  (FORCEM).  Like  most  theater-level  models  and  scenarios, 
FORCEM  is  a  low  resolution  expected  value  model,  representing  combat  forces  at  the  division  and 
higher  level  and  time  in  12-hour  steps.  The  Nuclear  Effects  Model  Embedded  Stochastically  in 
Simulation  (NEMESIS)  research  at  CAA  (Youngren  [1989])  documents  an  analytic  model  for 
describing  the  possible  outcomes  of  a  theater-level  tactical  nuclear  exchange.  The  methodology 
described  in  this  paper  arose  from  the  need  to  summarize  the  stochastic  outcomes  of  the  theater-level 
exchange  as  input  to  FORCEM. 

The  Scenario 

In  a  theater-level  battle  where  nuclear  weapons  may  be  employed,  the  commander  of  the  forces 
on  a  side  may  have  an  overall  objective  (such  as  stabilizing  the  forward  line  of  own  troops  (FLOT) 
in  the  defense  or  achieving  a  breakthrough  in  the  offense)  that  will  necessitate  the  use  of  nuclear 
weapons.  In  order  to  meet  this  objective,  the  commander  will  specify  the  defeat  criteria  against  each 
unit  —  that  is,  the  necessaty  degree  of  damage  to  be  achieved  against  each  unit  to  meet  his  objective. 
The  defeat  criteria  will  differ  from  unit  to  unit  depending  upon  the  unit  mission,  the  posture,  the 
equipment,  etc.  The  criteria  applied  to  larger  units  (such  as  divisions)  will  frequently  focus  fires  on 
critical  subordinate  units.  For  example,  the  defeat  criteria  for  a  unit  might  be  achieving  a  latent 
lethal  dose  (about  450  rad)  against  at  least  50  percent  of  the  personnel  in  the  unit.  The  defeat 
criteria  for  a  particular  division  might  be  to  defeat  at  least  50  percent  of  the  infantry  units  or  at 
least  40  percent  of  the  armor  units  in  the  division. 
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Although  the  effects  of  a  tactical  nuclear  laydown  at  the  theater  perspective  are  normally 
described  in  terms  of  defeating  divisions,  tactical  nuclear  weapons  within  the  theater  are  targeted 
against  forces  at  the  company  and  battery  level.  The  term  subunit  (also  target  or  target  subunit) 
used  in  this  paper  denotes  a  combat  organization  (such  as  a  company)  that  would  be  targeted  by  a 
nuclear  weapon.  The  size  of  the  subunit  will  depend  both  upon  the  capabilities  of  the  weapon  system 
used  to  engage  the  subunit  and  the  targeting  doctrine  of  the  firer.  For  example,  companies  may  be 
targeted  close  to  the  FLOT  using  small,  artillery-fired  weapons,  while  battalions  may  be  targeted 
deep  using  missiles  or  air-delivered  weapons.  For  purposes  of  exposition,  we  will  refer  to  the  low- 
resolution  combat  organizations  represented  in  theater  models  such  as  FORCEM  (usually  divisions, 
although  other  forces  may  be  represented  as  well)  as  units. 

There  are  very  many  targetable  subunits  in  a  typical  theater  scenario,  on  the  order  of  104.  As  a 
4 

result,  there  are  210  possible  outcomes  that  can  occur  in  terms  of  the  defeat  or  failure  to  defeat  each 

subunit.  Even  if  we  look  only  at  the  defeat  or  failure  to  defeat  the  low  resolution  aggregate  units 

2 

represented  in  our  theater  model  (usually  several  hundred),  we  still  have  on  the  order  of  210 
possible  outcomes.  Even  with  sophisticated  techniques  and  considerable  confounding,  classical 
experimental  design  approaches  require  at  least  one  run  per  variable.  The  large  amount  of  time  and 
effort  required  to  execute  even  a  simple  run  of  a  typical  theater-level  expected  value  model  prohibit- 
more  than  a  few  model  runs  for  any  study.  Classical  experimental  designs  therefore  obviously  cannot 
be  applied.  Our  objective  is  to  construct  a  plan  that  minimizes  the  number  of  different  input  data 
sets  (thus  minimizing  the  number  of  theater-level  model  runs)  yet  fully  reflects  the  range  of  possible 
outcomes  of  the  theater  nuclear  exchange. 

A  Method  for  Addressing  Input  Uncertainty  in  Expected  Value  Models 

Describing  the  outcome  of  the  theater-level  nuclear  exchange  on  each  unit  in  terms  of  defeat 
criteria  allows  us  to  define  a  binary  variable  B(,  where  B(-  =  1  if  the  unit  is  defeated;  0  otherwise. 
Given  the  assumption  that  the  outcome  is  independent  between  units,  the  outcome  of  any  exchange 
is  simply  a  set  of  0’s  and  l’s  with  the  probability  that  any  B;  =  1  equal  to  Pjeytat(»)t  the 
probability  that  unit  i  is  defeated,  i  —  1,  ...,  m.  Methods  for  easily  calculating  the  probability  of 
defeat  for  each  targetable  subunit  are  given  in  Youngren  [1989].  Given  m  units,  there  are  2m  possible 
outcomes.  Clearly,  if  we  define  defeat  criteria  in  terms  of  total  numbers  of  potential  nuclear  targets 
(on  the  order  of  104),  there  are  too  many  outcomes  to  enumerate. 
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At  the  theater  level,  however,  defeat  criteria  can  usually  be  expressed  in  terms  of  divisions  and  a 
limited  number  of  other  high  value  targets  --  on  the  order  of  at  most  several  hundred  across  a 
theater.  Each  division,  in  turn,  will  have  its  defeat  criteria  established  in  terms  of  units  subordinate 
to  that  division.  For  example,  suppose  that  a  division  j  has  10  battalions  of  infantry  (engaged  as 
battalions),  24  armored  companies  (engaged  as  companies),  and  20  batteries  of  artillery.  The  defeat 
criteria  for  this  division  may  be  50  percent  of  the  infantry,  40  percent  of  the  armor,  or  60  percent  of 
both,  with  a  separate  criteria  for  artillery  (divisional  and  nondivisional).  In  terms  of  maneuver 

subunits,  5  infantry  battalions  or  10  armor  companies  must  be  defeated  in  order  to  defeat  the 

(10+24)! 

division.  There  are  (i~Q_p)j  q!  (2~4 q~)!  wa^9  loosing  p  infantry  battalions  and  q  armored 

battalions  for  defeat,  and  all  combinations  where  p  >  5,  q  >  10,  or  (  p  +  q  )  >  60  percent  of  the 
subunit  (which  can  be  worked  out  for  specific  values  of  p  and  q  )  lead  to  the  defeat  of  this  division. 
If  we  assume  that  each  subunit  i,  i  =  1,  ...,  34  has  a  unique  probability  of  defeat  P^ea<(0i  we 
probably  do  not  wish  to  enumerate  all  sets  of  subunits  where  the  division  is  defeated  and  compute 
the  joint  probability  (which  will  be  the  product  of  P^ej>e4<(>)  for  the  subunits  i  defeated  and 
(1  — P^e/Ctt((1))  for  the  subunits  that  are  not).  Fortunately,  this  situation  is  readily  amenable  to 
Monte  Carlo  solutions.  We  simply  need  to  draw  34  binary  pseudorandom  numbers  B,-  such  that  each 
number  Bt-  =  1  with  probability  Pj4y44,(s)t  an^  a  binary  variable,  say  D„,  equal  1  if  the  set  of 
numbers  B,-  drawn  correspond  to  division  j  being  defeated,  0  otherwise.  If  we  perform  N  replications 
of  this  experiment,  we  can  estimate  P(  division  defeated  ]  =  h  £  Dn.  If  we  do  this  for  each  division 

1  n=l 

j,  then  we  have  a  probability  Pjejeat(div  j)  s  P[  division  j  defeated  ]  for  j  —  1,  ...,  ndiv,  where  ndiv 
—  the  number  of  divisions. 


At  the  division  level,  we  can  define  a  binary  variable  0;  to  define  the  outcome  of  the  nuclear 
exchange  with  respect  to  division  j,  j  =  1,  ndiv.  0;-  =  1  with  probability  P ^e/ea((^,v  j)  >f 
division  j  is  defeated;  0  otherwise. 


Across  the  theater,  the  theater  commander  will  desire  at  least  a  certain  percentage  of  units  be 

\ 

defeated  in  order  for  the  employment  of  nuclear  weapons  to  be  considered  effective.  We  can  define  a 
binary  function  of  the  random  variables  0,  <j>{  O  ),  such  that  <j>{  O  )  =  1  if  the  commander’s 
objective  is  met;  0  otherwise.  Clearly  <j>(  0  )  is  nondecreasing  in  O.  The  function  may  be  regarded 
as  identical  to  a  structure  function  of  a  coherent  system  in  reliability  theory  (Barlow  &;  Proschan 
[1981]);  thus  we  can  use  results  from  coherent  structure  theory  in  our  analysis  of  the  nuclear 
exchange  issue. 


6 


CAA-RP-89-5 


For  example,  if  any  k  out  of  m  divisions  must  be  defeated  in  order  for  the  commander’s 
objective  to  be  met, 

«Q)  =  (010J  (O,  o2 ...  0*.,  o4+1)ji  -l(0m.t+1  .-O*  ), 

for  all  possible  subsets  of  size  k  from  the  m  units,  1  <  it  <  m,  where 
(  xj  )  ji(  x4  )  =  1  -  (  1  -  x,  )(  1  —  Xj  ). 


Furthermore,  we  can  bound  P[  <j>(  0  )  =  1  ]  by  (Barlow  &  Proschan  [1981]  p.  31): 


max  ir  P[  0;  =  1  ]  <  P(  4>(  Q  )  =  1  ]  <  min  Jl  P[  0i  =  l  ]  , 

1  <  r  <  npaih  i£Pr  1  <  s  <  ncut  i£K, 

where  Pr  denotes  one  of  the  npaih  ™  'j  possible  min  path  sets  (in  this  case,  a  min  path  set  is 
any  set  of  k  units),  Kj  denotes  one  of  the  ncut  mlM-i  )  Poss^e  m'n  cut  9ets  (>n  lhis  case,  a 


min  cut  set  is  any  set  of  m-it+1  units),  and  _[]_  X,-  =  1  —  ”[F  ( 1  —  Xt).  If  we  let  p„(i)  = 

i  i 

P[  Oj  =  1  ],  and  number  the  units  such  that  Po(l)  <  Po(2)  <  •••  <  Po(m),  then 


max  "Tl 

1  <  r  <  npaih  i£pr 


p[  Oj  =  l  ] 


m 

TT  Pa(0 

i=m-k  +  l 


min 
<  s  < 


ncut 


JL  P[0;  =  1] 

«€A', 


m-t  +  l 

=  JL 


i=r/ 


Po(l’)  • 


This  example  of  a  k  out  on  m  defeat  criteria  shows  how  we  can  estimate  (through  bounds)  the 
probability  that  the  commander’s  objective  may  be  met..  Alternatively,  P[  <j>(  0  )  =  1  ]  can  be 
estimated  using  the  same  Monte  Carlo  technique  used  to  find  P[  0;-  =  1  ]  for  each  division  j. 


Partitioning  the  Space  of  All  Possible  Outcomes 


At  the  theater  level  with  a  total  of  nt  division-sized  and  high  value  targets,  if  we  examine  the 
nuclear  exchange  outcome  0^  for  each  division  (or  equivalent  high-value  target),  there  are  2U# 
possible  outcomes.  It  may  be  the  case  that  it  makes  a  difference  in  the  battle  that  follows  the 
nuclear  exchange  which  units  are  defeated  or  targets  destroyed  in  the  exchange.  Or,  more  simply,  it 
may  be  how  many  units  are  defeated  and  targets  destroyed  across  the  theater  which  makes  a 
difference. 


It  is  possible  to  define  sets  of  outcomes  of  the  nuclear  exchange  that,  given  our  best  judgment, 
we  expect  to  have  a  significantly  different  effect  on  any  subsequent  theater-level  battle  (if  all 
outcomes  have  approximately  the  same  effect,  then  there  is  one  set  consisting  of  all  outcomes).  We 
choose  these  sets  by  selecting  partitions  dividing  the  sample  space  (space  of  all  possible  outcomes) 
into  strata  such  that  the  following  properties  are  met: 
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(1)  All  events  within  a  given  stratum  will  yield  approximately  the  same  overall  theater-level 
outcome.  As  a  result  of  this  assumption,  we  regard  all  events  within  any  given  stratum  as 
exchangeable. 

(2)  Any  set  of  n  events  from  n  different  strata  are  expected  to  yield  n  different  theater-level 
outcomes.  Thus,  any  pair  of  events  from  two  different  strata  are  not  exchangeable. 

In  practice,  all  events  within  a  stratum  will  not  be  truly  exchangeable,  and  the  two  events  to 
either  “side”  of  any  partition  will  likely  lead  to  similar  theater-level  outcomes.  Nevertheless,  it  is 
possible  to  conceive  of  outcome  sets  with  different  results,  and  we  assume  for  all  of  the  development 
below  that  these  two  properties  are  obeyed. 


For  example,  suppose  that  there  are  20  opposing  divisions  in  a  sector  of  combat.  Our  best 
judgment,  given  the  tactical  and  operational  situation,  is  that  the  defeat  of  at  least  7  divisions  out  of 
the  20  will  be  required  to  avoid  loss  of  territory  (stabilize  the  FLOT— which  may  be  the 
commander’s  objective).  However,  if  14  or  more  divisions  are  defeated,  an  opportunity  occurs  not 
merely  to  stabilize  the  FLOT  but  also  to  conduct  a  successful  counterattack.  In  this  case,  if  Oj  =  1 
if  division  i  is  defeated,  i  —  1,  20,  there  are  220  possible  outcomes.  We  can  partition  the  sample 


space  of  possible  outcomes  into  the  ^  f  “i?  )  outcomes  where  6  or  fewer  divisions  are  defeated,  the 

o'  K  ' 

13  /  \  .  .  ♦  20  /  \ 

^  (  k  /  outcomes  w^ere  7  or  more  but  less  than  14  divisions  are  defeated,  and  the  ^  ^  J 


outcomes  where  14  or  more  divisions  are  defeated. 


The  example  given  above  involved  two  partitions  (three  strata);  the  number  of  partitions 
required  depends  on  the  number  of  significantly  different  theater-level  outcomes  that  need  to  be 
represented.  Selecting  the  partitions  will  require  experienced  judgment  and  possibly  some 
experimentation  with  the  theater  model.  If  one  is  unsure  about  how  many  partitions  to  select,  the 
number  of  strata  should  equal  the  maximum  number  of  theater  model  runs  you  can  afford. 


Stratified  Sampling  from  the  Sample  Space 


Once  the  sample  space  (space  of  all  possible  outcomes)  has  been  identified,  it  is  possible  to 
perform  a  stratified  sampling  from  the  sample  space,  each  sample  from  the  outcome  of  the  nuclear 
exchange  model  forming  an  input  vector  to  the  theater-level  conventional  model.  From  each  stratum 
created  by  our  partitions,  a  single  realization  can  be  sampled.  A  random  sampling  approach  can  be 
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used;  however,  since  the  actual  likelihood  of  all  of  the  events  within  a  stratum  may  vary  widely,  we 
recommend  using  a  fixed  sampling  scheme,  in  particular  sampling  the  mode  from  each  partition. 
Given  the  assumption  of  exchangeability  between  events  within  a  stratum,  any  choice  will  have  a 
roughly  equivalent  effect  on  the  theater-level  outcome,  so  any  choice  is  valid.  Using  the  mode  allows 
us  to  compensate  for  the  fact  that  the  events  within  the  stratum  are  only  approximately 
exchangeable.  A  modal  (most  likely)  outcome  will  also  form  a  plausible  input  suitable  for 
subsequent  analysis.  The  theater-level  conventional  model,  such  as  FORCEM,  will  be  run  ns  times 
for  each  of  the  ns  strata  created  from  ns— 1  partitions,  using  the  outcome  selected  from  each 
stratum  as  an  input.  If  the  second  assumption  that  we  made  in  selecting  the  partitions  is  met,  the  ns 
battles  simulated  in  FORCEM  using  outcomes  from  the  ns  different  strata  should  yield  noticeably 
different  results.  The  response  surface  estimated  using  these  ns  FORCEM  runs  should  provide  a 
better  representation  of  the  variability  possible  in  theater-level  combat  where  nuclear  weapons  are 
employed  than  a  random  selection  of  ns  outcomes  from  the  2n*  outcomes  possible,  where  nt  is  the 
number  of  targetable  subunits  in  the  theater. 

The  question  naturally  arises,  “what  if  I  am  wrong  in  selecting  the  partitions?”  Partitioning  is  a 
judgemental  process;  more  of  an  art  than  a  science.  The  situation  in  which  this  technique  is  to  be 
used  is  one  where  many  runs  of  the  deterministic  model  are  not  possible;  therefore,  it  is  not  possible 
to  sample  the  results  of  many  outputs  given  many  different  input  data  sets  describing  different 
nuclear  exchange  outcomes.  As  a  result,  we  simply  do  our  best  to  try  and  force  realizations  from 
areas  of  the  space  of  all  possible  outcomes  where  we  think  that  the  theater-level  outcome  will  be 
different.  The  impact  of  being  wrong  is  not  much  different  than  being  right.  We  still  have  another 
point  in  the  theater-level  outcome  space  that  you  are  sampling.  The  fact  that  the  nuclear  exchange 
outcome  did  not  lead  to  the  theater-level  outcome  expected  should  be  of  great  interest  to  the 
analysis.  Either  the  theater  model  has  deficiencies  in  correctly  representing  the  impact  of  the 
exchange,  or  the  theater  situation  is  (surprisingly)  robust  to  the  exchange.  If  the  theater  outcome 
that  you  tried  to  create  (by  selecting  the  nuclear  exchange  outcome  stratum)  is  still  of  interest, 
another  run  could  be  attempted  (if  time  and  resources  permit),  sampling  from  a  more  extreme  point 
within  the  stratum. 

Selecting  the  Most  Likely  Outcome  (Mode)  From  Each  Stratum 

Selecting  the  mode  from  each  stratum  is  simple  and  not  computationally  intensive.  The 
partitions  defining  the  stratum  will  establish  the  outcome  vectors  0  that  fall  within  each  stratum. 
Recall  that  p 0(j)  =  P[  Oj  =  1  ],  and  let  q 0(j)  —  1  —  p©0).  Order  the  p,^)  and  qo(j)’s  together 
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from  the  largest  to  the  smallest  value.  To  select  the  mode  within  each  partition,  go  from  the  first 
value  (po(j)  or  qo(j))  and  select  the  outcome  0 j  =  1  for  each  p0(j)  and  the  outcome  Oj  =  0  for 
each  q0(i).  Continue  until  each  target  j  has  an  outcome  assigned,  making  sure  to  assign  only  one 
outcome  to  each  target.  It  will  be  necessary  to  “skip”  over  the  higher  probability  ( p0(j)  or  q0(ji) )  for 
some  targets  j  in  order  to  have  a  total  set  of  outcomes  fall  within  the  partition. 

This  procedure  can  most  easily  be  understood  through  an  example.  Suppose  we  have  five 
divisional  units  with  the  following  probabilities  of  defeat  ( P[  0^  =  1  ]):  p0(l)  =  0.2,  p0(2)  —  0.25, 
p0(3)  —  p0(4)  =  0.4,  p0(5)  =  0.6.  We  also  have  the  following  strata  defined  in  terms  of  number  of 
units  defeated:  {  0,  1  },  {  2,  3,  4  },  and  {  5  }.  We  order  our  probabilities  as  follows:  q0(l)  =  0.8  > 
q«(2)  =  0.75  >  p0(5)  =  q,(3)  =  q.(4)  =  0.6  >  p0(3)  =  p,(4)  =  q,(5)  =  0.4  >  p0(2)  =  0.25  > 
Po(l)  =  0.2. 

The  first  stratum  must  have  zero  or  one  unit  defeated.  Thus  our  mode  for  the  first  stratum  is 
qo(l),qo(2)>po(5)-q0(3)-q0(4)  (i.e.,  outcomes  Ot=0,  O2=0,  05  =  1,  O3=0,  O4=0),  with  a 
probability  equal  to  (0.8)(0.75)(0.6)3  =  0.1296.  The  second  stratum  must  have  two,  three,  or  four 
units  defeated  and  the  mode  is  qo(l)-q<,(2)*p<,(5)-qo(3)-po(4),  with  a  probability  equal  to 
(0.8)(0.75)(0.6)2(0.4)  —  0.0864.  In  this  case,  we  “skipped”  outcome  O4“0  with  probability  0.6  and 
selected  outcome  C4sb1  with  probability  0.4  so  that  we  would  have  at  least  2  units  defeated  for  this 
strata.  Note  that  an  equally  likely  selection  would  be  q„(l)-q<.(2)-po(5)iqo(4)'Po(3).  The  third 
stratum  must  have  five  units  defeated  and  the  mode  is  po(5)-po(3)‘po(4)-p0(2)-po(l),  with  a 
probability  equal  to  (0.6)(0.4)J(0.25)(0.2)  =  0.0048. 

Interpreting  the  Results  of  Conventional  Runs  Using  Stratified  Inputs 

If  we  wish  to  obtain  an  output  measure  from  the  theater-level  conventional  model  that  we  wish 
to  average  across  all  possible  outcomes  (which  is  the  sort  of  thing  we  normally  do  in  our  simulation 
models),  we  need  to  construct  a  weighted  average  from  the  ns  runs  conducted  using  the  theater 
model.  The  weight  assigned  to  the  output  measure  from  each  run  k  would  be  the  total  likelihood  of 
all  events  within  stratum  k,  k  =  1,  ...,  ns.  If  it  is  possible  to  enumerate  all  of  the  possible  outcomes 
( nt  sufficiently  small),  this  likelihood  can  be  computed  directly.  If  ni  is  too  large,  we  can  conduct  a 
simple  Monte  Carlo  estimation  of  the  probability  pt  that  an  event  chosen  at  random  falls  within 
stratum  k ,  Jfc  =  1,  ...,  ns.  This  is  the  straightforward  process  of  estimating  the  vector  {  plt...,  pnl  } 
from  a  multinomial  distribution. 
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We  can  return  to  the  previous  example  to  illustrate  an  exact  computation  of  the  likelihood  of  all 
events  within  a  stratum.  Recall  that  the  strata  were  defined  in  terms  of  number  of  units  defeated: 
{  0,  1  },  {  2,  3,  4  },  and  {  5  }.  The  probability  that  0  units  are  defeated  is  P{  0  }  = 
qo(l),ql»(2)iq0(3)*q0(4)<ql,(5)  =  0.0864.  There  are  ^  ^  ^  =  5  possible  outcomes  leading  to  1  unit 
destroyed;  they  are: 

p*(l)*q.(2)*qo(3)-qo(4)*q®(5),  qo(l)-p<,(2)-qo(3)-q0(4)-q<>(5),  q,(l)«q*(2)-p„(3)-q,(4)-q<,(5), 

qo(l),qo(2)-q0(3),po(4)  *q<>(5),  qo(l)  •q«(2),q0(3)  •qo(4)-po(5) 
with  a  total  probability  of  0.0216+0.0288 +0.05764-0.0576+0. 1296  =  0.2952.  Thus  the  total 
likelihood  of  the  events  in  the  first  stratum  is  0.0864  +  0.2952  =  0.3816. 

The  calculations  for  P{2},  P{3},  and  P{4}  are  messy  (more  combinations)  but  straightforward. 
The  likelihoods  are  P{2}  =  0.3612,  P{3}  =  0.2012,  and  P{4)  =  0.0512,  for  a  total  likelihood  of 
0.6136.  The  likelihood  of  the  third  stratum  is  P{5}  =  0.0048. 

Adjustments 

In  practice,  several  cases  may  arise  where  it  is  desirable  to  make  some  adjustments  to  the  basic 
model.  We  describe  some  of  them  here. 

а.  Likelihood  of  any  realization  within  a  strata  being  too  small.  In  some  cases,  the  total 
likelihood  of  any  realization  from  a  particular  strata  may  be  too  small  to  justify  further 
consideration.  An  example  of  this  is  the  third  strata  ( {5} )  discussed  in  the  previous  paragraph.  A 
probability  of  less  than  0.01  is  likely  small  enough  to  ignore  in  our  theater  level  modeling  (this 
threshold  is,  of  course,  a  matter  of  judgment)  In  cases  such  as  this,  we  may  wish  to  simply  run  the 
conventional  theater  model  with  the  modes  from  the  more  likely  (in  the  example,  the  first  and 
second)  strata. 

б.  The  modes  from  two  strata  are  outcomes  that  are  adjacent  to  one  another.  It  is  possible  that 
the  modes  from  two  strata  are  at  the  boundary  of  their  respective  strata,  next  to  the  same  partition, 
and  thus  adjacent  to  one  another  in  terms  of  e.n  ordered  outcome  space.  An  example  of  this  is  also 
provided  in  the  previous  paragraph,  where  the  modes  from  the  first  two  strata  are  adjacent  to  one 
another  in  terms  of  units  defeated  (one  unit  defeated  in  the  first  stratum  and  two  in  the  second).  In 
order  to  reinforce  our  second  assumption  (different  results  from  different  strata),  we  may  wish  to 
make  a  different  selection  from  one  stratum  or  the  other  in  order  to  avoid  similar  results.  Two 
possible  adjustments  come  to  mind. 
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(1)  The  first  adjustment  is  to  select  the  next  highest  likelihood  from  within  either  stratum  that 
does  not  provide  the  same  number  of  units  defeated  as  does  the  mode.  In  our  example,  we  would 
choose  either  an  outcome  of  zero  units  defeated  from  the  first  stratum  or  three  or  four  units  defeated 
from  the  second  stratum.  The  most  likely  outcome  where  zero  units  are  defeated  is 
qo(l)-qo(2)-q0(3)-q<,(4)‘qtf(5)  =  0.0864.  The  most  likely  outcome  where  three  or  four  units  are 
defeated  is  q»(l)*qe(2)*p0(5)*po(3)‘po(4)  =  0.0576.  Since  0.0864  >  0.0576,  we  could  choose  the 
outcome  of  zero  units  defeated  from  the  first  stratum  and  keep  the  outcome  we  previously  computed 
(two  units  defeated)  for  the  second  stratum. 

(2)  The  second  possible  adjustment  is  to  define  partitions  such  that  there  are  “gaps”  between 
the  strata.  In  our  previous  example,  we  might  define  significantly  different  outcomes  coming  from 
zero  or  one  units  defeated,  three  or  four  defeated,  and  five  defeated,  where  the  outcome  of  two  units 
defeated  may  be  an  ambiguous  case  leading  to  either  the  same  result  as  {  0,  1  }  or  {  3,  4  }  defeated 
units.  This  approach  may  be  more  realistic,  as  the  “transitional  cases”  at  the  boundaries  of  the 
exhaustive  strata  may  lead  to  theater  outcomes  that  are  not  as  clear  cut  as  those  nearer  the  center  of 
any  particular  stratum.  The  only  drawback  to  this  approach  is  the  fact  that  the  total  likelihood  of 
drawing  results  from  any  of  the  strata  will  not  equal  one. 

Repeated  Exchanges 

Until  now,  we  have  assumed  that  there  is  essentially  only  one  nuclear  exchange  of  interest.  In 
other  words,  we  have  assumed  that  the  nuclear  weapons  will  be  employed  during  a  relatively  small 
timeframe  within  the  overall  theater  battle,  and  that  the  theater  battle  will  be  conventional 
thereafter  (at  least  for  the  duration  of  the  conflict  to  be  simulated).  However,  it  is  possible  that  a 
scenario  may  call  for  repeated  exchanges  of  nuclear  weapons.  We  can  handle  each  exchange  by 
defining  the  outcomes  through  binary  variables  and  stratifying  the  outcome  space  as  explained 
above.  However,  constructing  an  experimental  plan  with  a  reasonable  number  of  runs  of  the  theater 
model  becomes  difficult.  The  difficulty  rises  from  the  total  number  of  possible  combinations  of 
individual  exchange  outcomes,  even  if  only  a  few  strata  are  chosen  for  each  exchange.  For  example, 
only  three  exchanges  vith  only  three  significantly  different  outcomes  (strata)  predicted  per  exchange 
will  lead  to  33  =  27  different  possible  outcomes  after  all  three  exchanges.  It  is  probably  too 
expensive  to  execute  this  many  runs  of  a  theater-level  simulation  model. 

To  handle  such  a  situation,  we  begin  by  determining  the  probability  of  defeating  each  theater- 
level  unit  and  partitioning  the  set  of  all  possible  outcomes  as  explained  previously.  We  can  diagram 
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the  27  possible  outcomes  for  our  example  as  shown  below  in  Figure  1.  If  27  runs  are  too  many  to 
execute  on  our  theater  level  simulation,  then  we  must  select  a  smaller  subset  of  the  27  outcomes  to 
actually  use.  The  question  is,  of  course,  which  subset  do  we  pick?  A  stochastic  simulation  will 
randomly  select  paths  through  the  “tree”  (Figure  1)  by  selecting  individual  exchange  outcomes 
randomly  according  to  their  likelihoods.  When  a  stochastic  simulation  is  run  multiple  times,  the 
paths  with  a  high  probability  of  occurrence  will  be  selected  multiple  times  and  the  paths  with  a  low 
probability  of  occurrence  will  be  selected  infrequently  if  at  ail.  The  result  is  a  weighted  set  of 
outcomes  that  can  be  used  to  estimate  the  distribution  of  the  actual  outcome  after  three  exchanges. 
In  our  case,  we  cannot  even  afford  to  run  the  model  once  for  each  possible  outcome,  much  less 
multiple  times.  However,  we  have  the  same  objective  of  trying  to  determine  a  set  of  outcomes 
corresponding  to  particular  paths  that  can  be  weighted  to  estimate  the  distribution  of  the  actual 
outcome  after  three  exchanges. 


Figure  1.  Possible  Outcomes  from  Three  Exchanges  with  Three  Strata  Each 

Following  the  example  diagrammed  in  Figure  1,  let  us  label  the  9trata  at  each  exchange  as  high 
(H),  medium  (M),  and  low  (L)  corresponding  to  some  exchange  outcome  along  some  measure  (e.g., 
total  units  defeated).  We  can  bound  the  outcome  using  the  extreme  choices  at  each  decision  point  in 
our  tree;  i.e.,  HHH  for  an  upper  bound  and  LLL  for  a  lower  bound.  We  can  also  choose  an 
intermediate  outcome  (MMM)  in  this  case  by  choosing  the  intermediate  result  at  each  decision  point 
(note  that  there  may  not  always  be  a  clearly  defined  “middle”).  Beyond  this,  we  need  some  sort  of 
rationale  for  selecting  particular  outcomes  out  of  the  27  possible.  It  is  important  to  note  that  the 
variables  are  nested.  For  example,  the  middle  outcome  from  a  second  strike  following  a  high 
outcome  from  the  first  exchange  (HM)  will  be  different  from  the  middle  outcome  from  a  second 
strike  following  a  low  outcome  from  the  first  exchange  (LM),  because  the  force  strengths  surviving 
the  first  exchange  (and  thus  the  subsequent  theater  battle  before  the  second  exchange)  are 
significantly  different. 
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Several  approaches  come  to  mind,  both  qualitative  and  quantitative.  Qualitative  approaches  will 
choose  outcomes  according  to  the  strata;  for  example,  alternating  sequences  such  as  HML,  LMH,  and 
MLH  could  be  chosen. 

Quantitative  approaches  will  look  at  the  probability  assigned  to  each  stratum.  For  purposes  of 
illustration,  assume  that  the  probability  for  the  outcomes  (  H,  M,  L  )  are  (  .2,  .5  .3  )  respectively, 
and  that  the  probability  for  H,  M,  and  L  are  identical  for  each  of  the  three  exchanges  (in  reality, 
this  would  be  unlikely  but  it  suffices  for  illustration).  We  select  our  runs  according  to  their 
probabilities.  For  example,  the  most  likely  outcome  will  be  MMM  with  probability  (,5)3  =  0.125. 
The  next  most  likely  are  LMM,  MLM,  and  MML  with  probability  (.5)2(.3)  =  0.075,  etc.  We  can 
concentrate  on  choosing  the  outcomes  with  the  greatest  likelihood  (possibly  in  addition  to  the 
bounds  HHH  and  LLL). 

Interpreting  the  output  becomes  more  difficult  when  we  run  only  a  subset  of  all  possible 
outcome  strata.  In  our  standard  experimental  plan,  we  run  all  possible  outcome  strata  and  weight 
the  result  with  the  probability  associated  with  the  strata.  If  we  do  not  make  any  adjustments  (such 
as  defining  non-adjacent  strata),  the  probabilities  of  a  realization  coming  from  a  stratum  will  sum  to 
1.  When  we  select  a  subset  of  outcome  strata,  the  associated  probabilities  will  not  sum  to  1.  We 
recommend  normalizing  the  probabilities  associated  with  the  outcomes  selected  and  proceeding 
accordingly.  An  example  should  make  this  clear. 

Repeated  Exchanges  —  an  Example 

Suppose  we  have  three  exchanges  with  three  significantly  different  outcomes  (strata)  H,  M,  L 
with  probabilities  .2,  .5,  .3  respectively  as  stated  previously.  A  possible  selection  scheme  might  be 
the  following. 

(1)  Select  the  upper  and  lower  bounds  HHH  and  LLL.  The  associated  probabilities  are  HHH  = 
(.2)3  =  0.008  and  LLL  =  (.3)3  =  0.027. 

(2)  Select  the  middle  (qualitative)  or  modal  (quantitative)  outcome.  In  this  case,  they  are  the 
same  (MMM)  with  probability  (.5)3  =  0.125. 

(3)  Select  the  next  most  likely  outcomes  LMM,  MLM,  and  MML.  The  associated  probabilities 
are  equal  at  (.5)2(.3)  =  0.075.  Alternatively,  some  type  of  alternating  strata  sequence  could  be  used. 


14 


CAA-RP-89-5 


This  forms  a  subset  of  6  outcomes  out  of  the  27  possible.  The  total  probability  of  a  realization 
coming  from  any  of  the  6  selected  outcomes  is  0.008  +  0.027  +  0.125  +  (3)(0.075)  =  0.385.  The 
normalized  probabilities  are  therefore: 

HHH  =  °di  =  0  021 
UL  -  Sffi  = 

MMM  =  M  =  "•225 

LMM,  MLM,  MML  =  =  0.195. 

This  sums  to  1.001  due  to  rounding  error. 

In  this  example  we  would  execute  six  runs  of  the  theater-level  simulation  model,  selecting 
realizations  from  the  strata  associated  with  each  exchange  as  indicated  above  (for  example,  MLM 
would  select  from  the  middle  stratum  for  the  first  and  third  exchange,  and  the  lower  stratum  in  the 
second).  The  theater-level  model  output  associated  with  each  realization  selected  can  be  weighted 
with  the  normalized  probability  of  occurrence. 

Note  that  we  only  account  for  38.5  percent  of  the  possible  outcomes  in  terms  of  probability.  As 
a  result,  our  estimates  made  from  only  six  runs  will  not  be  as  good  as  those  produced  from  a  larger 
subset  from  the  27  possible. 

Averaging  the  Results 

To  continue  our  example,  suppose  that  an  outcome  for  some  particular  measure  from  a  theater 
conventional  model  such  as  FORCEM  was  125  for  a  run  using  input  from  the  first  stratum,  75  for  a 
run  from  the  second  stratum,  and  25  for  a  run  from  the  third  stratum.  An  average  value  for  this 
measure  would  be  derived  from  weighting  the  output  from  a  given  run  with  the  total  probability  of 
any  realization  coming  from  within  the  stratum.  In  our  example,  we  have  (125)(.3816)  + 
(75)(.6136)  +  (25)(.0048)  =  93.84.  This  value,  along  with  the  range  of  values  produced  by  the  three 
different  runs  (summarized  perhaps  with  a  weighted  variance  or  other  statistic),  should  be  much 
more  meaningful  than  the  value  obtained  by  running  FORCEM  only  for  some  arbitrarily  chosen 
input  set  for  the  nuclear  exchange  outcome. 

However,  a  word  of  caution  is  necessary.  We  started  with  the  assumption  that  there  is  more 
than  one  significantly  different  outcome  in  the  theater  context;  in  our  example,  there  were  three.  A 
single  summary  measure,  such  as  the  average,  does  not  reflect  this  reality.  Even  a  sample  average 
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and  variance  will  not  inform  a  decisionmaker  about  the  possible  outcomes  along  with  their 
associated  probabilities.  Since  the  total  number  of  runs  of  the  theater  conventional  model  will  be  (by 
necessity)  small,  we  recommend  reporting  all  of  the  results,  accompanied  perhaps  with  a  summary 
measure.  In  cases  of  tactical  nuclear  warfare,  we  are  often  concerned  with  relatively  unlikely  events 
(such  as  the  exchange  itself)  that  nevertheless  have  a  very  significant  impact.  Averaging  obscures 
this  fact  and  can  lead  a  decisionmaker  astray. 

Summary 

Using  a  deterministic,  expected  value  approach  to  model  a  real-world  situation  such  as  theater- 
level  combat  poses  problems  in  selecting  input  data.  A  deterministic  simulation  demands  a  single 
input  data  set  for  a  model  run,  while  the  data  may  have  to  represent  a  process  that  is  inherently 
stochastic.  An  example  is  provided  in  this  paper.  The  results  of  a  tactical  nuclear  exchange  within  a 
theater  is  inherently  stochastic,  driven  by  random  events  such  as  target  acquisitions.  An  “average” 
exchange  outcome  cannot  properly  be  defined;  an  average  fails  to  exist  in  subset  selection  problems 
(for  example,  if  20  units  out  of  50  are  acquired  on  the  average,  which  20  are  to  be  selected  as 
acquired  in  the  deterministic  model?)  Even  where  averages  can  be  defined,  they  fail  to  reflect 
important  variations  in  possible  outcomes  that  may  make  a  difference  between  winning  and  losing 
the  war  in  a  theater  simulation. 

Ideally,  a  theater-level  stochastic  model  would  be  used  to  properly  reflect  uncertainties  inherent 
in  .  the  data  and  processes  represented  by  the  model.  However,  the  current  state  of  the  art  in 
hardware  and  software  only  permit  us  (at  present)  to  model  combat  at  the  theater  in  a 
deterministic,  low-resolution  mode.  Thus,  we  must  reconcile  the  need  to  provide  an  input  to  these 
deterministic  models  with  the  reality  of  random  outcomes. 

4  in'* 

If  there  are  approximately  10  potential  nuclear  targets  in  a  theater,  there  are  2  possible 

outcomes  that  can  occur  in  terms  of  the  defeat  or  failure  to  defeat  each  potential  target.  Even  if  we 

look  only  at  the  defeat  or  failure  to  defeat  the  low  resolution  aggregate  units  represented  in  our 

2 

theater  model,  we  still  have  on  the  order  of  2 10  possible  outcomes.  A  classical  experimental  design 
approach  that  requires  at  least  one  run  per  variable  obviously  cannot  be  applied.  The  challenge, 
then,  is  to  construct  a  plan  that  minimizes  the  number  of  different  input  data  sets  yet  fully  reflects 
the  range  of  possible  outcomes  of  the  theater  nuclear  exchange. 
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This  paper  outlines  an  approach  to  constructing  such  an  experimental  plan.  We  begin  with  the 
probability  of  defeating  a  potential  nuclear  target  P^e8<(‘)  and  determine  from  that  the  probability 
of  defeating  the  aggregate  units  represented  in  our  theater  model  (such  as  divisions).  We  can 
characterize  all  possible  outcomes  of  the  exchange  as  sets  of  binary  variables,  where  each  binary 
variable  reflects  the  defeat  or  failure  to  defeat  each  unit.  We  then  partition  the  outcome  space  into 
strata  such  that  outcomes  from  different  strata  lead  to  significantly  different  results  in  the  theater 
battle,  and  all  significantly  different  outcomes  are  included  in  some  stratum.  Our  experimental  plan 
consists  of  a  nuclear  exchange  realization  from  each  strata  that  corresponds  to  the  most  likely 
outcome  within  that  stratum.  The  theater-level  model  is  run  using  the  experimental  plan  to 
determine  the  appropriate  input  data  set  to  use  to  reflect  the  outcome  of  a  theater  nuclear  exchange. 

Directions  for  Future  Research 

The  techniques  outlined  in  this  paper  form  only  a  start  at  trying  to  resolve  the  issue  of  how  to 
handle  uncertainty  in  input  to  large,  complex  expected  value  models.  They  are  presently  limited  to 
input  processes  that  can  be  summarized  in  a  reasonable  number  of  binary  variables,  where  it  is 
possible  to  make  a  judgement  about  the  type  of  expected  value  model  output  given  sets  of  similar 
input  realizations.  Nevertheless,  it  is  a  step  in  the  right  direction.  At  present,  it  is  not  infrequent  to 
find  studies  based  on  a  single  model  run  per  input  scenario,  without  any  estimate  of  the  variability 
possible  in  the  results  obtained. 

Possible  future  research  topics  include  extending  the  techniques  to  proceeses  that  can  be 
expressed  in  various  states,  the  number  of  such  states  exceeding  two.  Better  ways  of  estimating 
partitions  of  the  sample  space  may  also  be  developed.  A  very  realistic  case  in  many  theater  scenarios 
involves  repeated  realizations  of  random  processes  (in  the  context  of  the  nuclear  exchanges  discussed 
in  the  paper,  this  would  imply  many  small  weapon  exchanges  over  a  relatively  long  period  of  time). 
At  present,  we  have  no  satisfactory  way  of  handling  this  situation.  Robust  experimental  plans  that 
can  provide  meaningful  results  over  a  large  number  of  repeated  realizations  will  be  be  necessary  to 
model  such  scenarios. 
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APPENDIX  B 
NOTATION 

B(  A  binary  variable  denoting  the  defeat  /  failure  to  defeat  outcome  of  a  nuclear 

exchange  against  a  targetable  unit,  where  B(  =  1  if  the  unit  is  defeated;  0  otherwise. 

Dn  A  binary  variable  denoting  the  defeat  /  failure  to  defeat  outcome  of  a  nuclear 

exchange  against  a  division  for  the  nth  Monte  Carlo  replication,  where  D„  =  1  if  the 
set  of  numbers  B,-  drawn  correspond  to  a  division  being  defeated,  0  otherwise. 

H,M,L  A  qualitative  measure  of  the  nuclear  exchange  outcome  along  some  measure  (e.g., 

total  units  defeated).  H  stands  for  high,  M  for  medium,  and  L  for  low.  When  ordered, 
e.g.,  HLM,  the  first  letter  represents  the  outcome  of  the  first  exchange,  the  second  the 
outcome  of  the  second  exchange,  etc. 

K,  A  variable  which  denotes  the  sth  possible  min  cut  sets  (in  this  case,  a  min  cut  set  is 

any  set  of  m-i+l  units). 

ndiv  The  number  of  divisions. 

ns  The  number  of  strata  selected  for  a  particular  nuclear  exchange. 

nt  The  number  of  targetable  subunits  in  the  theater. 

Oj  A  binary  variable  used  to  define  the  outcome  of  the  nuclear  exchange  with  respect  to 

division  ;,  j  ~  1,  ndiv.  0;-  =  1  with  probability  Pjejeai(d‘v  j)  if  division  j  is 
defeated;  0  otherwise. 

<j>{  0  )  A  binary  function  of  the  random  variables  0,  defined  such  that  <t>(  Q  )  —  1  if  the 

commander’s  objective  (for  the  theater-wide  nuclear  exchange)  is  met;  0  otherwise. 

Pr  A  variable  which  denotes  the  rth  possible  min  path  sets  (in  this  case,  a  min  path  set 

is  any  set  of  k  units). 

p j)  Probability  that  division  j  defeated  ]  for  ;  =  1,  . . .,  ndiv. 
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Po(0 
(  x;.  )  ]i( 

JL 

i 

(5) 


The  probability  that  P[  0{  =  1  ];  that  is,  the  probability  that  division  »  is  defeated. 


)  For  2  units,  (  x,.  )  j[(  xf  )  =  1  -  (  1  -  x,-  )(  1  -  x;  ). 


For  n  units,  _[]_  X,  -  1  —If  (1— X<) 
i  i 


The  binomial  coefficient.  (  f*  \  is  defined  as  , , .  ,  ,, 

\  k  /  k!(n-k)! 
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